Search

Linear discriminant analysis (LDA) aims to find a low-dimensional space in which data points in the same class are to be close to each other while keeping data points from different classes apart. To improve the robustness of LDA to non-Gaussian distribution data, most existing discriminant analysis methods extend LDA by approximating the underlying manifold of data. However, these methods suffer from the following problems: 1) local affinity or reconstruction coefficients are learned on the basis of the relationships of all data pairs, which would lead to a sharp increase in the amount of computation and 2) they learn the manifold information in the original space, ignoring the interference of the noise and redundant features. Motivated by these challenges, this article represents a novel discriminant analysis model, called fast and adaptive locality discriminant analysis (FALDA), to improve the efficiency and robustness. First, with the anchor-based strategy, a bipartite graph of each class is constructed to characterize the local structure of data. Since the number of anchor points is far less than that of data points, learning of fuzzy membership relationships between data points and anchor points within each class can save training time. Second, a maximum entropy regularization is introduced to control the uniformity of the weights of graphs and avoid the trivial solution. Third, the above relationships are updated adaptively in the process of dimensionality reduction, which can suppress the interference of the noise and redundant features. Fourth, the whitening constraint is imposed on the projection matrix to remove the relevance between features and restrict the total scatter of data in the subspace. Last but not the least, data with complex distribution can be explicitly divided into sub-blocks according to the learned anchor points (or subclass center points). We test our proposed method on synthetic data, benchmark datasets, and imbalanced datasets. Promising experimental results demonstrate the success of this novel model.

It is hard to construct an optimal classifier for high-dimensional imbalanced data, on which the performance of classifiers is seriously affected and becomes poor. Although many approaches, such as resampling, cost-sensitive, and ensemble learning methods, have been proposed to deal with the skewed data, they are constrained by high-dimensional data with noise and redundancy. In this study, we propose an adaptive subspace optimization ensemble method (ASOEM) for high-dimensional imbalanced data classification to overcome the above limitations. To construct accurate and diverse base classifiers, a novel adaptive subspace optimization (ASO) method based on adaptive subspace generation (ASG) process and rotated subspace optimization (RSO) process is designed to generate multiple robust and discriminative subspaces. Then a resampling scheme is applied on the optimized subspace to build a class-balanced data for each base classifier. To verify the effectiveness, our ASOEM is implemented based on different resampling strategies on 24 real-world high-dimensional imbalanced datasets. Experimental results demonstrate that our proposed methods outperform other mainstream imbalance learning approaches and classifier ensemble methods.

Learning-to-learn, a progressive speedup of learning while solving a series of similar problems, represents a core process of knowledge acquisition that draws attention in both neuroscience and artificial intelligence. To investigate its underlying brain mechanism, we trained a recurrent neural network model on arbitrary sensorimotor mappings known to depend on the prefrontal cortex. The network displayed an exponential time course of accelerated learning. The neural substrate of a schema emerges within a low-dimensional subspace of population activity; its reuse in new problems facilitates learning by limiting connection weight changes. Our work highlights the weight-driven modifications of the vector field, which determines the population trajectory of a recurrent network and behavior. Such plasticity is especially important for preserving and reusing the learnt schema in spite of undesirable changes of the vector field due to the transition to learning a new problem; the accumulated changes across problems account for the learning-to-learn dynamics.

Partial multilabel learning (PML) aims to learn from training data, where each instance is associated with a set of candidate labels, among which only a part is correct. The common strategy to deal with such a problem is disambiguation, that is, identifying the ground-truth labels from the given candidate labels. However, the existing PML approaches always focus on leveraging the instance relationship to disambiguate the given noisy label space, while the potentially useful information in label space is not effectively explored. Meanwhile, the existence of noise and outliers in training data also makes the disambiguation operation less reliable, which inevitably decreases the robustness of the learned model. In this article, we propose a prior label knowledge regularized self-representation PML approach, called PAKS, where the self-representation scheme and prior label knowledge are jointly incorporated into a unified framework. Specifically, we introduce a self-representation model with a low-rank constraint, which aims to learn the subspace representations of distinct instances and explore the high-order underlying correlation among different instances. Meanwhile, we incorporate prior label knowledge into the above self-representation model, where the prior label knowledge is regarded as the complement of features to obtain an accurate self-representation matrix. The core of PAKS is to take advantage of the data membership preference, which is derived from the prior label knowledge, to purify the discovered membership of the data and accordingly obtain more representative feature subspace for model induction. Enormous experiments on both synthetic and real-world datasets show that our proposed approach can achieve superior or comparable performance to state-of-the-art approaches.

With the continuously increasing amount of video data, image set classification has recently received widespread attention in the CV & PR community. However, the intra-class diversity and inter-class ambiguity of representations remain an open challenge. To tackle this issue, several methods have been put forward to perform multiple geometry-aware image set modelling and learning. Although the extracted complementary geometric information is beneficial for decision making, the sophisticated computational paradigm (e.g., scatter matrices computation and iterative optimisation) of such algorithms is counterproductive. As a countermeasure, we propose an effective hybrid Riemannian metric learning framework in this paper. Specifically, we design a multiple graph embedding-guided metric learning framework for the sake of fusing these complementary kernel features, obtained via the explicit RKHS embeddings of the Grassmannian manifold, SPD manifold, and Gaussian embedded Riemannian manifold, into a unified subspace for classification. Furthermore, the involved optimisation problem of the developed model can be solved in terms of a series of sub-problems, achieving improved efficiency theoretically and experimentally. Substantial experiments are carried out to evaluate the efficacy of our approach. The experimental results suggest the superiority of it over the state-of-the-art methods.

Multivariate time-series (MTS) clustering is a fundamental technique in data mining with a wide range of real-world applications. To date, though some approaches have been developed, they suffer from various drawbacks, such as high computational cost or loss of information. Most existing approaches are single-view methods without considering the benefits of mutual-support multiple views. Moreover, due to its data structure, MTS data cannot be handled well by most multiview clustering methods. Toward this end, we propose a consistent and specific non-negative matrix factorization-based multiview clustering (CSMVC) method for MTS clustering. The proposed method constructs a multilayer graph to represent the original MTS data and generates multiple views with a subspace technique. The obtained multiview data are processed through a novel non-negative matrix factorization (NMF) method, which can explore the view-consistent and view-specific information simultaneously. Furthermore, an alternating optimization scheme is proposed to solve the corresponding optimization problem. We conduct extensive experiments on 13 benchmark datasets and the results demonstrate the superiority of our proposed method against other state-of-the-art algorithms under a wide range of evaluation metrics.

Multiview subspace clustering (MVSC) leverages the complementary information among different views of multiview data and seeks a consensus subspace clustering result better than that using any individual view. Though proved effective in some cases, existing MVSC methods often obtain unsatisfactory results since they perform subspace analysis with raw features that are often of high dimensions and contain noises. To remedy this, we propose a self-guided deep multiview subspace clustering (SDMSC) model that performs joint deep feature embedding and subspace analysis. SDMSC comprehensively explores multiview data and strives to obtain a consensus data affinity relationship agreed by features from not only all views but also all intermediate embedding spaces. With more constraints being cast, the desirable data affinity relationship is supposed to be more reliably recovered. Besides, to secure effective deep feature embedding without label supervision, we propose to use the data affinity relationship obtained with raw features as the supervision signals to self-guide the embedding process. With this strategy, the risk that our deep clustering model being trapped in bad local minima is reduced, bringing us satisfactory clustering results in a higher possibility. The experiments on seven widely used datasets show the proposed method significantly outperforms the state-of-the-art clustering methods. Our code is available at https://github.com/kailigo/dmvsc.git.

Complex systems can be defined by “sloppy” dimensions, meaning that their behavior is unmodified by large changes to specific parameter combinations, and “stiff” dimensions whose change results in considerable behavioral modification. In the neocortex, sloppiness in synaptic architectures would be crucial to allow for the maintenance of asynchronous irregular spiking dynamics with low firing rates despite a diversity of inputs, states, and both short- and long-term plasticity. Using simulations on neural networks with first-order spiking statistics matched to firing in murine visual cortex while varying connectivity parameters, we determined the stiff and sloppy parameters of synaptic architectures across three classes of input (brief, continuous, and cyclical). Algorithmically-generated connectivity parameter values drawn from a large portion of the parameter space reveal that specific combinations of excitatory and inhibitory connectivity are stiff and that all other architectural details are sloppy. Stiff dimensions are consistent across input classes with self-sustaining synaptic architectures following brief input occupying a smaller subspace as compared to the other input classes. Experimentally estimated connectivity probabilities from mouse visual cortex are consistent with the connectivity correlations found and fall in the same region of the parameter space as architectures identified algorithmically. This suggests that simple statistical descriptions of spiking dynamics are a sufficient and parsimonious description of neocortical activity when examining structure-function relationships at the mesoscopic scale. Additionally, coarse graining cell types does not prevent the generation of accurate, informative, and interpretable models underlying simple spiking activity. This unbiased investigation provides further evidence of the importance of the interrelationship of excitatory and inhibitory connectivity to establish and maintain stable spiking dynamical regimes in the neocortex.Author SummaryConnections between neurons are continuously changing to allow learning and adaptation to new stimuli. However, the ability of neural networks to vary these connections while avoiding excessively high- or low-activity states is still not well understood. We tackled this question by studying how changes in the parameters of connectivity within and between different neuronal populations impacted network activity in computational models. We identified specific combinations of parameters, deemed “stiff”, that must be maintained to observe activity consistent with recordings from murine visual cortex, while the rest of the parameters can be varied freely with minimal effects on activity. Our results agree with experimentally measured connectivity statistics demonstrating the importance of balancing opposing forces to maintain activity in a natural regime.

Feature selection has attracted a lot of attention in obtaining discriminative and non-redundant features from high-dimension data. Compared with traditional filter and wrapper methods, embedded methods can obtain a more informative feature subset by fully considering the importance of features in the classification tasks. However, the existing embedded methods emphasize the above importance of features and mostly ignore the correlation between the features, which leads to retain the correlated and redundant features with similar scores in the feature subset. To solve the problem, we propose a novel supervised embedded feature selection framework, called feature selection under global redundancy minimization in orthogonal regression (GRMOR). The proposed framework can effectively recognize redundant features from a global view of redundancy among the features. We also incorporate the large margin constraint into GRMOR for robust multi-class classification. Compared with the traditional embedded methods based on least square regression, the proposed framework utilizes orthogonal regression to preserve more discriminative information in the subspace, which can help accurately rank the importance of features in the classification tasks. Experimental results on twelve public datasets demonstrate that the proposed framework can obtain superior classification performance and redundancy removal performance than twelve other feature selection methods.

High-dimensional data are highly correlative and redundant, making it difficult to explore and analyze. Amount of unsupervised dimensionality reduction (DR) methods has been proposed, in which constructing a neighborhood graph is the primary step of DR methods. However, there exist two problems: 1) the construction of graph is usually separate from the selection of projection direction and 2) the original data are inevitably noisy. In this article, we propose an unsupervised adaptive embedding (UAE) method for DR to solve these challenges, which is a linear graph-embedding method. First, an adaptive allocation method of neighbors is proposed to construct the affinity graph. Second, the construction of affinity graph and calculation of projection matrix are integrated together. It considers the local relationship between samples and global characteristic of high-dimensional data, in which the cleaned data matrix is originally proposed to remove noise in subspace. The relationship between our method and local preserving projections (LPPs) is also explored. Finally, an alternative iteration optimization algorithm is derived to solve our model, the convergence and computational complexity of which are also analyzed. Comprehensive experiments on synthetic and benchmark datasets illustrate the superiority of our method.

Facing with rapidly increasing demands for analyzing high-order data or multiway data, feature-extracting methods become imperative for analysis and processing. The traditional feature-extracting methods, however, either need to overly vectorize the data and smash the original structure hidden in data, such as PCA and PCA-like methods, which is unfavorable to the data recovery, or cannot eliminate the redundant information very well, such as tucker decomposition (TD) and TD-like methods. To overcome these limitations, we propose a more flexible and more powerful tool, called the multiview principal components analysis (Multiview-PCA) in this article. By segmenting a random tensor into equal-sized subarrays called sections and maximizing variations caused by orthogonal projections of these sections, the Multiview-PCA finds principal components in a parsimonious and flexible way. In so doing, two new operations on tensors, the S -direction inner/outer product, are introduced to formulate tensor projection and recovery. With different segmentation ways characterized by section depth and direction, the Multiview-PCA can be implemented many times in different ways, which defines the sequential and global Multiview-PCA, respectively. These multiple Multiview-PCA take the PCA and PCA-like, and TD and TD-like as the special cases, which correspond to the deepest section depth and the shallowest section depth, respectively. We propose an adaptive depth and direction selection algorithm for the implementation of Multiview-PCA. The Multiview-PCA is then tested in terms of subspace recovery ability, compression ability, and feature extraction performance when applied to a set of artificial data, surveillance videos, and hyperspectral imaging data. All numerical results support the flexibility, effectiveness, and usefulness of Multiview-PCA.

Linear discriminant analysis (LDA) is a classical statistical machine-learning method, which aims to find a linear data transformation increasing class discrimination in an optimal discriminant subspace. Traditional LDA sets assumptions related to the Gaussian class distributions and single-label data annotations. In this article, we propose a new variant of LDA to be used in multilabel classification tasks for dimensionality reduction on original data to enhance the subsequent performance of any multilabel classifier. A probabilistic class saliency estimation approach is introduced for computing saliency-based weights for all instances. We use the weights to redefine the between-class and within-class scatter matrices needed for calculating the projection matrix. We formulate six different variants of the proposed saliency-based multilabel LDA (SMLDA) based on different prior information on the importance of each instance for their class(es) extracted from labels and features. Our experiments show that the proposed SMLDA leads to performance improvements in various multilabel classification problems compared to several competing dimensionality reduction methods. publishedVersion

Sharp speed variation leads to a shift of sample distribution domain, which poses a challenge for vibration-based rolling bearing fault diagnosis. Furthermore, the overfitting effects inflicted on the intelligent diagnosis model due to insufficient data will hinder the performance significantly. In this work, a Subspace Network with Shared Representation learning (SNSR) based on meta-learning is constructed for fault diagnosis under speed transient conditions with few samples. Firstly, shared representation learning based on the cross mutual information estimation is designed to promote the encoder to learn the domain invariant features. Meanwhile, we developed non-parameterized adaptive weight allocation to optimize the estimation of the discriminator. Then, the subspace classifiers in the meta-learning paradigm are employed to force the encoder to learn the discriminative features. Finally, the shared representation learning is embedded into the meta-learning and a cross co-training mechanism is designed for optimization. Thus the fusion framework is endowed with the capacity of learning distinguishable and domain invariant features simultaneously for diagnosis under speed transient conditions with few samples. Comparative experiments on two case studies of bearing fault diagnosis validated the superior performance of the proposed method, with an accuracy of 97.72% and 96.46% in 7-way and 9-way learning respectively.

Due to the shortage of COVID-19 viral testing kits, radiology is used to complement the screening process. Deep learning methods are promising in automatically detecting COVID-19 disease in chest x-ray images. Most of these works first train a Convolutional Neural Network (CNN) on an existing large-scale chest x-ray image dataset and then fine-tune the model on the newly collected COVID-19 chest x-ray dataset, often at a much smaller scale. However, simple fine-tuning may lead to poor performance due to two issues, firstly the large domain shift present in chest x-ray datasets and secondly the relatively small scale of the COVID-19 chest x-ray dataset. In an attempt to address these issues, we formulate the problem of COVID-19 chest x-ray image classification in a semi-supervised open set domain adaptation setting and propose a novel domain adaptation method, Semi-supervised Open set Domain Adversarial network (SODA). SODA is designed to align the data distributions across different domains in the general domain space and also in the common subspace of source and target data. In our experiments, SODA achieves a leading classification performance compared with recent state-of-the-art models in separating COVID-19 with common pneumonia. We also present results showing that SODA produces better pathology localizations.

In most of the existing representation learning frameworks, the noise contaminating the data points is often assumed to be independent and identically distributed (i.i.d.), where the Gaussian distribution is often imposed. This assumption, though greatly simplifies the resulting representation problems, may not hold in many practical scenarios. For example, the noise in face representation is usually attributable to local variation, random occlusion, and unconstrained illumination, which is essentially structural, and hence, does not satisfy the i.i.d. property or the Gaussianity. In this article, we devise a generic noise model, referred to as independent and piecewise identically distributed (i.p.i.d.) model for robust presentation learning, where the statistical behavior of the underlying noise is characterized using a union of distributions. We demonstrate that our proposed i.p.i.d. model can better describe the complex noise encountered in practical scenarios and accommodate the traditional i.i.d. one as a special case. Assisted by the proposed noise model, we then develop a new information-theoretic learning framework for robust subspace representation through a novel minimum weighted error entropy criterion. Thanks to the superior modeling capability of the i.p.i.d. model, our proposed learning method achieves superior robustness against various types of noise. When applying our scheme to the subspace clustering and image recognition problems, we observe significant performance gains over the existing approaches.

In this letter, an improved model predictive control (MPC) has been proposed for single-phase grid-tied converter with virtual vectors in a reshaped and compacted solution space, while improving ac power quality, without increase of the virtual vectors number. Combining linear solution space reconstruction method, the global optimization solution space can be shrunken and reshaped in real-time, only relying on grid angle information. Therefore, MPC can select the optimal virtual-vector in a finer and closer solution subspace, then, much finer control effect can be obtained. Implemented by finite-control-set MPC with an integrated difference modulator, equivalent fixed double switching frequency at the output can be achieved. Experimental results validate the effectiveness of the proposed methods.

Ridge regression is frequently utilized by both supervised learning and semisupervised learning. However, the results cannot obtain the closed-form solution and perform manifold structure when ridge regression is directly applied to semisupervised learning. To address this issue, we propose a novel semisupervised feature selection method under generalized uncorrelated constraint, namely SFS. The generalized uncorrelated constraint equips the framework with the elegant closed-form solution and is introduced to the ridge regression with embedding the manifold structure. The manifold structure and closed-form solution can better save data's topology information compared to the deep network with gradient descent. Furthermore, the full rank constraint of the projection matrix also avoids the occurrence of excessive row sparsity. The scale factor of the constraint that can be adaptively obtained also provides the subspace constraint more flexibility. Experimental results on data sets validate the superiority of our method to the state-of-the-art semisupervised feature selection methods.

Precision assembly is one of the primary goals of robots in manufacturing. Assembly strategies combining active and passive compliant control are presented herein. We construct a high-dimensional configuration space of robot assembly and then divide the configuration space to its subspaces. We further map the active compliant motion and the passive compliant motion of manipulator into different subspaces. In one subspace, we construct the constraint function and design the passive compliant motion of manipulator in the constraint region, where the uncertainties of the system should be eliminated by the environment constraints. In another subspace, we design a force controller based on the low-resolution force sensory information to control the position of the robot. The proposed method avoids the design of precision mechanism systems and the usage of high-quality sensors. Several experiments pertaining to peg-in-hole insertions are conducted to demonstrate the efficiency of the proposed method.

Lately, variational autoencoder (VAE) has arisen as one of the most prevalent methods for unsupervised learning of intricate distributions. Despite being successful in deep feature extraction and uncertain data modelling, it still suffers from the instability and reconstruction error due to random sampling in the latent subspace representation of original input space. In this paper, to deal with those limitations, constrained VAE (CVAE) is proposed by utilizing input sample information. Enthused by parallel interaction mechanism between the ventral and dorsal stream of human brain in object recognition, parallel interaction spatial-temporal CVAE (PIST-CVAE) is proposed to extract spatial and temporal features from input samples. Lower dimensional nonlinear features extracted from PIST-CVAE are used to build the soft sensor. The effectiveness of CVAE and PIST-CVAE is demonstrated in an industrial case study, a polyester polymerization process.

We propose a novel structured analysis-synthesis dictionary pair learning method for efficient representation and image classification, referred to as relaxed block-diagonal dictionary pair learning with a locality constraint (RBD-DPL). RBD-DPL aims to learn relaxed block-diagonal representations of the input data to enhance the discriminability of both analysis and synthesis dictionaries by dynamically optimizing the block-diagonal components of representation, while the off-block-diagonal counterparts are set to zero. In this way, the learned synthesis subdictionary is allowed to be more flexible in reconstructing the samples from the same class, and the analysis dictionary effectively transforms the original samples into a relaxed coefficient subspace, which is closely associated with the label information. Besides, we incorporate a locality-constraint term as a complement of the relaxation learning to enhance the locality of the analytical encoding so that the learned representation exhibits high intraclass similarity. A linear classifier is trained in the learned relaxed representation space for consistent classification. RBD-DPL is computationally efficient because it avoids both the use of class-specific complementary data matrices to learn discriminative analysis dictionary, as well as the time-consuming l₁/l₀-norm sparse reconstruction process. The experimental results demonstrate that our RBD-DPL achieves at least comparable or better recognition performance than the state-of-the-art algorithms. Moreover, both the training and testing time are significantly reduced, which verifies the efficiency of our method. The MATLAB code of the proposed RBD-DPL is available at https://github.com/chenzhe207/RBD-DPL.

Linear discriminant analysis (LDA) is a well-known technique for supervised dimensionality reduction and has been extensively applied in many real-world applications. LDA assumes that the samples are Gaussian distributed, and the local data distribution is consistent with the global distribution. However, real-world data seldom satisfy this assumption. To handle the data with complex distributions, some methods emphasize the local geometrical structure and perform discriminant analysis between neighbors. But the neighboring relationship tends to be affected by the noise in the input space. In this research, we propose a new supervised dimensionality reduction method, namely, locality adaptive discriminant analysis (LADA). In order to directly process the data with matrix representation, such as images, the 2-D LADA (2DLADA) is also developed. The proposed methods have the following salient properties: 1) they find the principle projection directions without imposing any assumption on the data distribution; 2) they explore the data relationship in the desired subspace, which contains less noise; and 3) they find the local data relationship automatically without the efforts for tuning parameters. The performance of dimensionality reduction shows the superiorities of the proposed methods over the state of the art.

Evolutionary multitasking (EMT) with the ability to tackle multiple different tasks has attracted more and more attention. The transferred knowledge among tasks can simultaneously improve the solving efficiency of all optimization problems in the evolutionary process. However, if the way of knowledge transfer is inappropriate, the negative knowledge transfer will make a bad effect on the performance of EMT algorithms. It is worth studying that how to promote the positive knowledge transfer across tasks. In this paper, a multiobjective multitasking algorithm named EMT-DAVT is introduced. The proposed algorithm contains two components, namely subspace distribution alignment (DA) strategy and decision variable transfer (VT) mechanism. In DA strategy, a learned mapping matrix is utilized to align the distributions in the subspaces and reduces the divergence between subpopulations belonging to different tasks. Then, VT mechanism is implemented which can further promote the positive information transfer. The two proposed strategies interact with each other to improve the knowledge transfer. Finally, the search engine is designed to balance exploration and exploitation. The experimental results on the multiobjective multitasking test suites demonstrate that EMT-DAVT outperforms other classical multiobjective multitasking evolutionary algorithms.

This paper develops data-driven methods to identify eigenfunctions of the Koopman operator associated to a dynamical system and subspaces that are invariant under the operator. We build on Extended Dynamic Mode Decomposition (EDMD), a data-driven method that finds a finite-dimensional approximation of the Koopman operator on the span of a predefined dictionary of functions. We propose a necessary and sufficient condition to identify Koopman eigenfunctions based on the application of EDMD forward and backward in time. Moreover, we propose the Symmetric Subspace Decomposition (SSD) algorithm, an iterative method which provably identifies the maximal Koopman-invariant subspace and the Koopman eigenfunctions in the span of the dictionary. We also introduce the Streaming Symmetric Subspace Decomposition (SSSD) algorithm, an online extension of SSD that only requires a small, fixed memory and incorporates new data as is received. Finally, we propose an extension of SSD that approximates Koopman eigenfunctions and invariant subspaces when the dictionary does not contain sufficient informative eigenfunctions.

This paper considers a problem of estimating brain effective connectivity from EEG signals using a Granger causality (GC) concept characterized on state-space models. We propose a state-space model for explaining coupled dynamics of the source and EEG signals where EEG is a linear combination of sources according to the characteristics of volume conduction. Our formulation has a sparsity prior on the source output matrix that can further classify active and inactive sources. The scheme is comprised of two main steps: model estimation and model inference to estimate brain connectivity. The model estimation consists of performing a subspace identification and the active source selection based on a group-norm regularized least-squares. The model inference relies on the concept of state-space GC that requires solving a discrete-time Riccati equation for the covariance of estimation error. We verify the performance on simulated data sets that represent realistic human brain activities under several conditions including percentages of active sources, a number of EEG electrodes and the location of active sources. The performance of estimating brain networks is compared with a two-stage approach using source reconstruction algorithms and VAR-based Granger analysis. Our method achieved better performances than the two-stage approach under the assumptions that the true source dynamics are sparse and generated from state-space models. The method is applied to a real EEG SSVEP data set and we found that the temporal lobe played a role of a mediator of connections between temporal and occipital areas, which agreed with findings in previous studies.

As an effective method for clustering applications, the clustering ensemble algorithm integrates different clustering solutions into a final one, thus improving the clustering efficiency. The key to designing the clustering ensemble algorithm is to improve the diversities of base learners and optimize the ensemble strategies. To address these problems, we propose a clustering ensemble framework that consists of three parts. First, three view transformation methods, including random principal component analysis, random nearest neighbor, and modified fuzzy extension model, are used as base learners to learn different clustering views. A random transformation and hybrid multiview learning-based clustering ensemble method (RTHMC) is then designed to synthesize the multiview clustering results. Second, a new random subspace transformation is integrated into RTHMC to enhance its performance. Finally, a view-based self-evolutionary strategy is developed to further improve the proposed method by optimizing random subspace sets. Experiments and comparisons demonstrate the effectiveness and superiority of the proposed method for clustering different kinds of data.

This study develops a novel data-driven latent variable correlation analysis (LVCA) framework to achieve communication efficient distributed monitoring for industrial plant-wide processes. Process data of a local unit are first projected into a dominant latent variable subspace and a residual subspace to characterize the correlation within the local unit. Then, least absolute shrinkage and selection operator is used to determine communication variables from neighboring units that are beneficial for monitoring the local unit. Thereafter, canonical correlation analysis is performed between the dominant subspace and communication variables to characterize the correlation between units. Finally, a distributed monitor is established for each unit, which considers the correlation within the local unit and the correlation between different operation units. The proposed LVCA-based distributed monitoring scheme is applied on a numerical example, the Tennessee Eastman benchmark process, and a lab-scale distillation process. Comparison results with some state-of-the-art methods verify the effectiveness.

Frequency selective surface (FSS) is critical for electromagnetic (EM) radiation protection due to its high spatial filtering performance, especially for active FSS. Recently, artificial neural network (ANN) has shown great potential in solving EM inverse problems and rapid industrial design. In such an inverse model with ANN, it establishes the relationship between the given inputs of S-parameters and the desired structure parameters or material parameters. However, faced with applications where S-parameters vary in a large frequency range with different curve shapes, such as multi-band microwave devices, equal interval sampling may result in high dimensional inputs and will require more complicated neural network. In this work, we present a Fourier subspace-based deep learning method (FS-BDLM) for FSS inverse design, where the dimension of the input is largely reduced by using Fourier subspace to represent the most salient features of the desired S-parameter performance. Compared with existing deep learning methods, the proposed technique makes inverse neural models more compact and more stable to noise contaminations. The validation of the proposed FS-BDLM is conducted both numerically and experimentally through two dual-passband FSS design examples, where the well-designed FSS is fabricated to validate the technique.

The searching ability of the population-based search algorithms strongly relies on the coordinate system on which they are implemented. However, the widely used coordinate systems in the existing multifactorial optimization (MFO) algorithms are still fixed and might not be suitable for various function landscapes with differential modalities, rotations, and dimensions; thus, the intertask knowledge transfer might not be efficient. Therefore, this article proposes a novel intertask knowledge transfer strategy for MFOs implemented upon an active coordinate system that is established on a common subspace of two search spaces. The proper coordinate system might identify some common modality in a proper subspace to some extent. In this article, to seek the intermediate subspace, we innovatively introduce the geodesic flow that starts from a subspace, reaching another subspace in unit time. A low-dimension intermediate subspace is drawn from a uniform distribution defined on the geodesic flow, and the corresponding coordinate system is given. The intertask trial generation method is applied to the individuals by first projecting them on the low-dimension subspace, which reveals the important invariant features of the multiple function landscapes. Since intermediate subspace is generated from the major eigenvectors of tasks' spaces, this model turns out to be intrinsically regularized by neglecting the minor and small eigenvalues. Therefore, the transfer strategy can alleviate the influence of noise led by redundant dimensions. The proposed method exhibits promising performance in the experiments.

Multiview subspace clustering has been demonstrated to achieve excellent performance in practice by exploiting multiview complementary information. One of the strategies used in most existing methods is to learn a shared self-expressiveness coefficient matrix for all the view data. Different from such a strategy, this article proposes a rank consistency induced multiview subspace clustering model to pursue a consistent low-rank structure among view-specific self-expressiveness coefficient matrices. To facilitate a practical model, we parameterize the low-rank structure on all self-expressiveness coefficient matrices through the tri-factorization along with orthogonal constraints. This specification ensures that self-expressiveness coefficient matrices of different views have the same rank to effectively promote structural consistency across multiviews. Such a model can learn a consistent subspace structure and fully exploit the complementary information from the view-specific self-expressiveness coefficient matrices, simultaneously. The proposed model is formulated as a nonconvex optimization problem. An efficient optimization algorithm with guaranteed convergence under mild conditions is proposed. Extensive experiments on several benchmark databases demonstrate the advantage of the proposed model over the state-of-the-art multiview clustering approaches.

Currently, visual-tactile fusion learning for robotic object recognition has achieved appealing performance, due to the fact that visual and tactile data can offer complementary information. However: 1) the distinct gap between vision and touch makes it difficult to fully explore the complementary information, which would further lead to performance degradation and 2) most of the existing visual-tactile fused learning methods assume that visual and tactile data are complete, which is often difficult to be satisfied in many real-world applications. In this article, we propose a partial visual-tactile fused (PVTF) framework for robotic object recognition to address these challenges. Specifically, we first employ two modality-specific (MS) encoders to encode partial visual-tactile data into two incomplete subspaces (i.e., visual subspace and tactile subspace). Then, a modality gap mitigated (MGM) network is adopted to discover modality-invariant high-level label information, which is utilized to generate gap loss and further help updating the MS encoders for relatively consistent visual and tactile subspaces generation. In this way, the huge gap between vision and touch is mitigated, which would further contribute to mine the complementary visual-tactile information. Finally, to achieve data completeness and complementary visual-tactile information exploration simultaneously, a cycle subspace leaning technique is proposed to project the incomplete subspaces into a complete subspace by fully exploiting all the obtainable samples, where complete latent representations with maximum complementary information can be learned. A lot of comparative experiments conducted on three visual-tactile datasets validate the advantage of the proposed PVTF framework, by comparing with state-of-the-art baselines.

Neighborhood reconstruction is proved effective for dimensionality reduction because of the preservation of manifold structure. Conventional neighborhood preserving embedding (NPE) method first learns the affinity relationship or reconstruction relationship in the original space, and then learns the projection matrix to preserve the learned local information in low-dimensional space. However, the pre-learned manifold information may be inaccurate due to the noises and irrelevant features in real-world data. The performance of dimensionality reduction would be influenced as well. Besides, NPE and its variants only aim to preserve the local reconstruction relationship but ignore the fuzzy membership relationship between samples and cluster prototypes. To address these issues, we propose an adaptive neighborhood preserving discriminant projection model, where sparse reconstruction coefficients are updated in the process of dimensionality reduction to eliminate the influence of noises and irrelevant features. Meanwhile, we also learn the fuzzy membership relationships between data points and cluster prototypes to gather the samples belonging to the same class together in low-dimensional space. Neighborhood reconstruction learning and clustering are seamlessly connected in the learned subspace. To solve this model, an iterative algorithm is developed. The experimental results of recognition accuracy show the superiorities of the proposed methods over the state-of-the-arts.

Supervised dimensionality reduction for sequence data learns a transformation that maps the observations in sequences onto a low-dimensional subspace by maximizing the separability of sequences in different classes. It is typically more challenging than conventional dimensionality reduction for static data, because measuring the separability of sequences involves non-linear procedures to manipulate the temporal structures. In this paper, we propose a linear method, called order-preserving Wasserstein discriminant analysis (OWDA), and its deep extension, namely DeepOWDA, to learn linear and non-linear discriminative subspace for sequence data, respectively. We construct novel separability measures between sequence classes based on the order-preserving Wasserstein (OPW) distance to capture the essential differences among their temporal structures. Specifically, for each class, we extract the OPW barycenter and construct the intra-class scatter as the dispersion of the training sequences around the barycenter. The inter-class distance is measured as the OPW distance between the corresponding barycenters. We learn the linear and non-linear transformations by maximizing the inter-class distance and minimizing the intra-class scatter. In this way, the proposed OWDA and DeepOWDA are able to concentrate on the distinctive differences among classes by lifting the geometric relations with temporal constraints. Experiments on four 3D action recognition datasets show the effectiveness of OWDA and DeepOWDA.

In real-world applications, complete or incomplete multi-view data are common, which leads to the problem of generalized multi-view clustering. Recently, researchers attempt to learn the latent representation in the common subspace from heterogeneous data, which usually suffers from feature degeneration. Moreover, there are limited efforts on simultaneously revealing the underlying subspace structure and exploring the complementary information from incomplete multiple views. In this paper, we introduce a novel Generalized Multi-view Collaborative Subspace Clustering (GMCSC) framework to address the above issues, in which consensus subspace structure of all views and embedding subspaces for each view are jointly learned to benefit each other. Specifically, we develop a novel collaborative subspace learning strategy based on self-representation learning, which provides a brand-new way of pursuing the complete subspace structure directly from multi-view data. Furthermore, we explore complementary information by enforcing the consistency across different views and preserving the view-specific information of each view, which can alleviate the problem of feature degeneration and enhance the reasonability of using a consensus representation for multiple views. Experimental results on six benchmark datasets demonstrate that the proposed method can significantly outperform the state-of-the-art algorithms.

Speech emotion recognition has become an attractive research topic due to various emotional states of speech signals in real-life scenarios. Most current speech emotion recognition methods are carried out on a single corpus. However, in practice, the training and testing data often come from different domains, e.g., different corpora. In this case, the model generalizability and recognition performance would decrease greatly due to the domain mismatch. To address this challenging problem, we present a transfer learning method, called joint transfer subspace learning and regression (JTSLR), for cross-corpus speech emotion recognition. Specifically, JTSLR performs transfer subspace learning and regression in a joint framework. First, we learn a latent subspace by introducing a discriminative maximum mean discrepancy (MMD) as the discrepancy metric. Then, we put forward a regression function in this latent subspace to describe the relationships between features and corresponding labels. Moreover, we present a label graph to help transfer knowledge from relevant source data to target data. Finally, we conduct extensive experiments on three popular emotional datasets. The results show that our method can outperform traditional methods and some state-of-the-art transfer learning algorithms for cross-corpus speech emotion recognition tasks.

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging l2,1-norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the l2,0-norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S²DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks.

In data sets some attributes may have higher or lower importance. One of the tools used for data analysis of such datasets are subspace neuro-fuzzy systems. They elaborate fuzzy rules to describe data sets. In subspace neuro-fuzzy systems fuzzy rules exist in subspaces defined with subsets of attributes. In the paper we propose a novel fuzzy biclustering algorithm that groups both objects and attributes in fuzzy clusters. In that way we create a subspace fuzzy rule base for a subspace fuzzy system. The paper is accompanied with numerical examples that show this approach can lead to better generalisation (and thus lower data prediction errors) with preserved interpretation of fuzzy models.

Human action recognition which recognizes human actions in a video is a fundamental task in computer vision field. Although multiple existing methods with single-view or multi-view have been presented for human action recognition, these recognition approaches cannot be extended into new action recognition or action classification tasks, as well as discover underlying correlations among different views. To tackle the above problem, this paper proposes a new lifelong multi-view subspace learning framework for continuous human action recognition, which could exploit the complementary information amongst different views from a lifelong learning perspective. More specifically, a set of view-specific libraries is established to gradually store the useful information within multiple views. As a new action recognition task comes, we decompose the model parameters into a set of embedded parameters over view-specific libraries. A latent representation subspace is constructed via encouraging it to be close to different view-specific libraries, which can leverage the high-order correlations among different views and further avoid partial information for action recognition task. Meanwhile, we propose to employ an alternating direction strategy to optimize our proposed method. Empirical studies on real-world multi-view action recognition datasets have shown that our proposed framework attains the superior recognition performance and saves the computational time when continually learning new action recognition tasks.

Surrogate-based-constrained optimization for some optimization problems involving computationally expensive objective functions and constraints is still a great challenge in the optimization field. Its difficulties are of two primary types. One is how to handle the constraints, especially, equality constraints; another is how to sample a good point to improve the prediction of the surrogates in the feasible region. Overcoming these difficulties requires a reliable constraint-handling method and an efficient infill-sampling strategy. To perform inequality- and equality-constrained optimization of expensive black-box systems, this work proposes a hybrid surrogate-based-constrained optimization method (HSBCO), and the main innovation is that a new constraint-handling method is proposed to map the feasible region into the origin of the Euclidean subspace. Thus, if the constraint violation of an infeasible solution is large, then it is far from the origin in the Euclidean subspace. Therefore, all constraints of the problem can be transformed into an equivalent equality constraint, and the distance between an infeasible point and the origin in the Euclidean subspace represents the constraint violation of the infeasible solution. Based on the distance, the objective function of the problem can be penalized by a Gaussian penalty function, and the original constrained optimization problem becomes an unconstrained optimization problem. Thus, the feasible solutions of the original minimization problem always have a lower objective function value than any infeasible solution in the penalized objective space. To improve the optimization performance, kriging-based efficient global optimization (EGO) is used to find a locally optimal solution in the first phase of HSBCO, and starting from this locally optimal solution, RBF-model-based global search and local search strategies are introduced to seek global optimal solutions. Such a hybrid optimization strategy can help the optimization process converge to the global optimal solution within a given maximum number of function evaluations, as demonstrated in the experimental results on 23 test problems. The method is shown to achieve the global optimum more closely and efficiently than other leading methods.

This article presents a novel discriminative subspace-learning-based unsupervised domain adaptation (DA) method for the gas sensor drift problem. Many existing subspace learning approaches assume that the gas sensor data follow a certain distribution such as Gaussian, which often does not exist in real-world applications. In this article, we address this issue by proposing a novel discriminative subspace learning method for DA with neighborhood preserving (DANP). We introduce two novel terms, including the intraclass graph term and the interclass graph term, to embed the graphs into DA. Besides, most existing methods ignore the influence of the subspace learning on the classifier design. To tackle this issue, we present a novel classifier design method (DANP+) that incorporates the DA ability of the subspace into the learning of the classifier. The weighting function is introduced to assign different weights to different dimensions of the subspace. We have verified the effectiveness of the proposed methods by conducting experiments on two public gas sensor datasets in comparison with the state-of-the-art DA methods.

Multiview clustering (MVC) has recently been the focus of much attention due to its ability to partition data from multiple views via view correlations. However, most MVC methods only learn either interfeature correlations or intercluster correlations, which may lead to unsatisfactory clustering performance. To address this issue, we propose a novel dual-correlated multivariate information bottleneck (DMIB) method for MVC. DMIB is able to explore both interfeature correlations (the relationship among multiple distinct feature representations from different views) and intercluster correlations (the close agreement among clustering results obtained from individual views). For the former, we integrate both view-shared feature correlations discovered by learning a shared discriminative feature subspace and view-specific feature information to fully explore the interfeature correlation. This allows us to attain multiple reliable local clustering results of different views. Following this, we explore the intercluster correlations by learning the shared mutual information over different local clusterings for an improved global partition. By integrating both correlations, we formulate the problem as a unified information maximization function and further design a two-step method for optimization. Moreover, we theoretically prove the convergence of the proposed algorithm, and discuss the relationships between our method and several existing clustering paradigms. The experimental results on multiple datasets demonstrate the superiority of DMIB compared to several state-of-the-art clustering methods.

In this paper, a novel time-domain frequency estimation approach is proposed based on the well-known Subspace Identification Method (SIM). Differing from the Fast Fourier Transformation (FFT) method, the nonlinear identification for frequency estimation is reformulated into the eigenvalue identification problem, which avoids the spectrum peak search. The recursive frequency estimation approach is proposed based on the updating/downdating of the Cholesky decomposition, where the extended observability matrix can be identified from a small matrix with lower computation cost. In addition, a gap metric-oriented performance indicator is proposed as a test statistic for fault detection as well as the evaluation of fault severity. The effectiveness of the proposed methods is verified for frequency estimation and fault diagnosis performance through numerical simulations and the experimental measurements from a laboratory unmanned aerial vehicle (UAV) rig with partial blade damage.

This paper considers a subspace guarding game in high-dimensional space which consists of a play subspace and a target subspace. Two faster defenders cooperate to protect the target subspace by capturing an attacker which strives to enter the target subspace from the play subspace without being captured. A closed-form solution is provided from the perspectives of kind and degree. Contributions of the work include the use of the attack subspace (AS) method to construct the barrier, by which the game winner can be perfectly predicted before the game starts. In addition to this inclusion, with the priori information about the game result, a critical payoff function is designed when the defenders can win the game. Then, the optimal strategy for each player is explicitly reformulated as a saddle-point equilibrium. Finally, we apply these theoretical results to a half-space guarding game in three-dimensional space. Since the whole achieved developments are analytical, they require a little memory without computational burden and allow for real-time updates, beyond the capacity of traditional Hamilton-Jacobi-Isaacs method. It is worth noting that this is the first time in the current work to consider the target guarding games for arbitrary high-dimensional space, and in a fully analytical form., Comment: 12 pages, 2 figures

In this article, considering the supervised dimensionality reduction, we first propose a model, called infinite Bayesian max-margin linear discriminant projection (iMMLDP), by assembling a set of local regions, where we make use of Bayesian nonparametric priors to handle the model selection problem, for example, the underlying number of local regions. In each local region, our model jointly learns a discriminative subspace and the corresponding classifier. Under this framework, iMMLDP combines dimensionality reduction, clustering, and classification in a principled way. Moreover, to deal with more complex data, for example, a local nonlinear separable structure, we extend the linear projection to a nonlinear case based on the kernel trick and develop an infinite kernel max-margin discriminant projection (iKMMDP) model. Thanks to the conjugate property, the parameters in these two models can be inferred efficiently via the Gibbs sampler. Finally, we implement our models on synthesized and real-world data, including multimodally distributed datasets and measured radar image data, to validate their efficiency and effectiveness.

With the increased proliferation of smart devices, the transit passengers of today expect a higher quality of service in the form of real-time traffic updates, accurate expected time-of-arrival (ETA) predictions. Providing these services requires public transit agencies and private transportation players to maintain full situational awareness of the city-wide traffic. However, most such agencies and companies are resource-constrained and do not have access to city-wide traffic data. The availability of sparsely sampled and outlier-corrupted traffic data renders the resulting traffic maps patchy and unreliable and necessitates the use of sophisticated real-time traffic interpolation and prediction algorithms. Moreover, since the traffic data is measured and collected in a sequential manner, the estimations must also be generated online. Thankfully, the traffic matrices are spatially and temporally structured, allowing the use of time-series and matrix/tensor completion algorithms. This work puts forth a generative model for the traffic density and subsequently uses a variational Bayesian formalism to learn the parameters of the model. Specifically, we consider low-rank traffic matrices whose subspace evolves according to a state-space model with possible sparse outliers. Unlike most matrix/tensor completion algorithms, the proposed model is equipped with automatic relevance determination priors that allow it to learn the parameters in an entirely data-driven manner. A forward-backward algorithm is proposed that enables the updates to be carried out at low-complexity. Simulations carried out on real traffic speed data demonstrate that the proposed algorithm better predicts the future traffic densities as compared to the state-of-the-art matrix/tensor completion algorithms.

We show that the projection of a product of two or three Eisenstein series of level one onto the cuspidal subspace is not an eigenform unless the dimension of the cuspidal subspace is one.

Multiview subspace learning has attracted much attention due to the efficacy of exploring the information on multiview features. Most existing methods perform data reconstruction on the original feature space and thus are vulnerable to noisy data. In this article, we propose a novel multiview subspace learning method, called multiview consensus structure discovery (MvCSD). Specifically, we learn the low-dimensional subspaces corresponding to different views and simultaneously pursue the structure consensus over subspace clustering for multiple views. In such a way, latent subspaces from different views regularize each other toward a common consensus that reveals the underlying cluster structure. Compared to existing methods, MvCSD leverages the consensus structure derived from the subspaces of diverse views to better exploit the intrinsic complementary information that well reflects the essence of data. Accordingly, the proposed MvCSD is capable of producing a more robust and accurate representation structure which is crucial for multiview subspace learning. The proposed method can be optimized effectively, with theoretical convergence guarantee, by alternatively iterating the argument Lagrangian multiplier algorithm and the eigendecomposition. Extensive experiments on diverse datasets demonstrate the advantages of our method over the state-of-the-art methods.

This article develops an identification algorithm for nonlinear systems. Specifically, the nonlinear system identification problem is formulated as a sparse recovery problem of a homogeneous variant searching for the sparsest vector in the null subspace. An augmented Lagrangian function is utilized to relax the nonconvex optimization. Thereafter, an algorithm based on the alternating direction method and a regularization technique is proposed to solve the sparse recovery problem. The convergence of the proposed algorithm can be guaranteed through theoretical analysis. Moreover, by the proposed sparse identification method, redundant terms in nonlinear functional forms are removed and the computational efficiency is thus substantially enhanced. Numerical simulations are presented to verify the effectiveness and superiority of the present algorithm.